XR Adaptive Modality: Case Study
What I Built
React + TypeScript experiment UI
Responsive Fitts' Law task + HUD, two modalities, pressure manipulation, and block-level workload collection.
Psychophysics-inspired gaze proxy
Fixation-state Gaussian jitter, saccadic suppression, and confirmation behavior to reproduce gaze failure modes in remote testing.
Policy engine + replayable diagnostics
Threshold triggers with hysteresis, policy state logging, and replay summaries to verify when adaptation did (or didn't) execute.
Reproducible reporting pipeline
Mixed-effects models + LBA cognitive modeling + Quarto-based reproducible analysis. All figures and tables regenerate from raw logs with consistent QC and model specifications.
Implementation notes (optional) — for engineers
Development approach: Despite not being a software/web developer by training, I delivered this full-stack research platform efficiently using Cursor AI for code generation and iteration, enabling rapid prototyping and deployment. The entire system—from React UI to analysis pipeline—was built and deployed to Vercel in a matter of weeks, demonstrating that modern AI-assisted development tools can bridge domain expertise gaps.
Tech stack:
- Frontend: React 18 + TypeScript, Vite build tool, React Router for multi-page flow
- Deployment: Vercel (automatic CI/CD from GitHub, zero-config deployment)
- Analysis: R (tidyverse, lme4, emmeans) + Quarto for reproducible reporting
- Data export: CSV download + optional EmailJS submission (compressed with pako)
Architecture components:
- Task UI (
app/src/components/FittsTask.tsx): Canvas-based ISO 9241-9 compliant Fitts’ Law implementation with circular target layout, real-time movement tracking, and hit/miss detection - Gaze proxy (
app/src/lib/modality.ts): Psychophysics simulation with fixation-state Gaussian jitter, saccadic suppression (velocity-based freezing), and confirmation model (dwell/SPACE + tolerance ring) - Policy engine (
app/src/lib/policy.ts): Rule-based adaptation with RT percentile thresholds, error burst detection, hysteresis (N-trial smoothing), and replayable state logs for debugging - Telemetry collector (
app/src/lib/telemetry/collector.ts): Structured event logging (trial lifecycle, movement samples at 60Hz, display metadata, policy state changes) with automatic QC flagging (focus/blur, invalid RTs, display violations) - Multi-page flow: React Router with routes for consent → demographics → system check → calibration → task → debrief
System Overview:
- Stack: React 18 + TypeScript, deployed on Vercel
- Experiment Engine: Block/condition assignment using Williams counterbalancing sequences
- Deployment: Vercel with automated CI/CD
Telemetry Schema:
The telemetry system logs comprehensive per-trial data including trial metadata, performance metrics, Fitts’ parameters, endpoint data, movement samples (60Hz), display state, and policy state. Key fields include:
- Trial Metadata: trial_id, session_id, participant_id, taskType
- Performance Metrics: rt_ms, correct, movement_time_ms, submovement_count
- Fitts’ Parameters: A (amplitude), W (width), ID (index of difficulty), target_width_px
- Endpoint Data: endpoint_x/y, endpoint_error_px, target_center_x/y, path_length_px, peak_velocity_px_s, curvature_index
- Movement Samples: 60Hz sampling with sd_along_px, sd_ortho_px for path analysis
- Display State: screen_width, screen_height, device_pixel_ratio
- Policy State: adaptation_triggered (boolean), width_scale_factor (applied scaling)
Key technical decisions:
- Client-side only: All processing happens in the browser (no backend), enabling fast deployment and privacy-preserving data collection
- Event-driven architecture: Global event bus for inter-component communication (TaskPane ↔︎ HUDPane ↔︎ LoggerPane)
- Policy replay system: Every policy decision is logged with full context, enabling post-hoc analysis of “why adaptation did/didn’t trigger”
- Reproducible analysis: Quarto document (
Report.qmd) regenerates all figures/tables from raw CSV logs with consistent QC exclusions
File structure highlights:
app/src/routes/— Multi-page flow components (Intro, Demographics, SystemCheck, Calibration, Task, Debrief)app/src/components/— Reusable UI components (FittsTask, TaskPane, HUDPane, TLXForm)app/src/lib/— Core logic (policy engine, modality simulation, telemetry, CSV export)policy/— JSON policy configuration files (default vs locked variants)analysis/— R scripts for data processing, QC, and statistical modelingReport.qmd— Main Quarto analysis document (regenerates all outputs)
Performance & optimization:
- 60Hz movement sampling for smooth cursor tracking
- Compressed telemetry export (pako gzip) for EmailJS size limits
- Lazy loading and code splitting via Vite
- TypeScript strict mode for type safety and early error detection